Towards innovation measurement in software industry

Context: In today’s highly competitive business environment, shortened product and technology life-cycles, it is critical for software industry to continuously innovate. To help an organisation to achieve this goal, a better understanding and control of the activities and determinants of innovation is required. This can be achieved through innovation measurement initiative which assesses innovation capability, output and performance. Objective: This study explores definitions of innovation, innovation measurement frameworks, key elements of innovation and metrics that have been proposed in literature and used in industry. The degree of empirical validation and context of studies was also investigated. It also elicited the perception of innovation, its importance, challenges and state of practice of innovation measurement in software industry. Methods: In this study, a systematic literature review, followed by online questionnaire and face-to-face interviews were conducted. The systematic review used seven electronic databases, including Compendex, Inspec, IEEE Xplore, ACM Digital Library, and Business Source premier, Science Direct and Scopus. Studies were subject to preliminary, basic and advanced criteria to judge the relevance of papers. The online questionnaire targeted software industry practitioners with different roles and firm sizes. A total of 94 completed and usable responses from 68 unique firms were collected. Seven face-to-face semi-structured interviews were conducted with four industry practitioners and three academics. Results: Based on the findings of literature review, interviews and questionnaire a comprehensive definition of innovation was identified which may be used in software industry. The metrics for evaluation of determinants, inputs, outputs and performance were aggregated and categorised. A conceptual model of the key measurable elements of innovation was constructed from the findings of the systematic review. The model was further refined after feedback from academia and industry through interviews. Conclusions: The importance of innovation measurement is well recognised in both academia and industry. However, innovation measurement is not a common practice in industry. Some of the major reasons are lack of available metrics and data collection mechanisms to measure innovation. The organisations which do measure innovation use only a few metrics that do not cover the entire spectrum of innovation. This is partly because of the lack of consistent definition of innovation in industry. Moreover, there is a lack of empirical validation of the metrics and determinants of innovation. Although there is some static validations, full scale industry trials are currently missing. For software industry, a unique challenge is development of alternate measures since some of the existing metrics are inapplicable in this context. The conceptual model constructed in this study is one step towards identifying measurable key aspects of innovation to understanding the innovation capability and performance of software firms.